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‘Terms? Information, entropy, redunc ney, uncertainty, average 
inforuntion, transmitted informition or corzrol^tion monsure, 
informational correspondence, stechastic processes, ergodic, 
correlational redundancy, Cistribution:1 reduncancy, positive 
interaction uncertainty, higher order redundancy (digrens, 
trigrams, ctc.), isomorphism, iconicity, abstraction, distrotion, 


noise, relative entropy, multidimensional scaling, = . o 
Basic relation: i - 
H = > p,logp, (Shannon) 
; Relative Entropy: Re H/H ax = H/iogín) 
nedundancy : C = 1-R 


Shannon's conriunicotion model: 


Information Transmitter Receiver Destination 
Source 


Message 
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Ae Review of basic facts about logarithms: 


Primary fact: if, N= xP then p = log, N .- 
Cporations: 
xP) (xI) = ¿Pta xP/49 = „Pd 
logM = logM + logN logM/W = logM ~ logN 
Log? = plogM logh fff = 1/p(109M) 
Know the meaning of: mantissa and characteristic, 


B. Using log vith base 2, Assumed in information theorys 


Basic fact: log, 92 = 0,30103 
Deduce: 109,10 = (Leg, 92)! = 3,322 i 
“xercises in log with base 2: 


log dj ,%,0,1,2,4,8,16,32,64,128,256,512,1024) = 


Problems: Throo piace 
answers: 
log (1000) 24 | 9.966 
log (10°) ay 19,932 
aes 6,644 
1 160) 7.322 
103 (3.16)4" 1.661 
1og(2,5)4* 1.322 
log (5 )ar 2.322 
log(s3/2)^49 / 
leg (25 a 4.644 
2oduce 10g(3)4* 1,585 
log) ar L " 
leg (80) % 6.322 
2oduce log(3)* 1.585 
log (50)% 5.644 
Deduce log(7)4= 2,807 


Caleulate the log of the integers between 1 -n4 5E 
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